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Intellectual Property Rights 



IPRs essential or potentially essential to the present document may have been declared to ETSI. The information 
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found 
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in 
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web 
server ( http://webapp.etsi.org/IPR/home.asp ). 

Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee 
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web 
server) which are, or may be, or may become, essential to the present document. 



Foreword 

This Technical Specification (TS) has been produced by ETSI Technical Committee GRID (GRID). 
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Scope 



The present document defines a grid testing framework based on existing testing and validation methodologies, best 
practices and tools used, in IT and Telecom sectors to obtain ICT Interoperability. 



References 



References are either specific (identified by date of publication and/or edition number or version number) or 
non-specific. 

• For a specific reference, subsequent revisions do not apply. 

• Non-specific reference may be made only to a complete document or a part thereof and only in the following 

cases: 

if it is accepted that it will be possible to use all future changes of the referenced document for the 
purposes of the referring document; 

for informative references. 

Referenced documents which are not found to be publicly available in the expected location might be found at 
http://docbox.etsi.org/Reference . 

NOTE: While any hyperlinks included in this clause were valid at the time of publication ETSI cannot guarantee 
their long term validity. 

2.1 Normative references 

The following referenced documents are indispensable for the application of the present document. For dated 
references, only the edition cited applies. For non-specific references, the latest edition of the referenced document 
(including any amendments) applies. 

Not applicable. 

2.2 Informative references 

The following referenced documents are not essential to the use of the present document but they assist the user with 
regard to a particular subject area. For non-specific references, the latest version of the referenced document (including 
any amendments) applies. 

[i.l] ETSI EG 202 237 (VI. 1.2): "Methods for Testing and Specification (MTS); Internet Protocol 

Testing (IPT); Generic approach to interoperability testing". 

[i.2] ETSI TS 102 828 (VI. 1.1): "GRID; Grid Component Model (GCM); GCM Apphcation 

Description". 

[i.3] ETSI TS 102 827 (VI. 1.1): "GRID; Grid Component Model (GCM); GCM Interoperability 

Deployment" . 

[i.4] ETSI TS 102 829 (VI. 1.1): "GRID; Grid Component Model (GCM);GCM Fractal Architecture 

Description Language (ADL)". 

[i.5] IBM, Workload Management with LoadLeveler, IBM Redbooks. 

NOTE: Available at http://www.redbooks.ibm.com/abstracts/sg246038.html . 

[i.6] Platform, Manage and accelerate compute- or data-intensive workload on HPC clusters and grids. 

NOTE: Available at http://www.platform.com/Products/platform-lsf 
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[i.7] Microsoft, Microsoft Windows Compute Cluster Server 2003 (CCS). 

NOTE: Available at http://www.niicrosoft.com/windowsserver2003/ccs/default.mspx . 

[i.8] PBS Gridworks, OpenPBS. 

NOTE: Available at: 

http://www.pbsgridworks.eom/PBSTempl.3.aspx7top nav name=Products&item name=OpenPBS&top 
nav str= 1 &AspxAutoDetectCookieSupport= 1 . 

[i.9] Sun, Sun Grid Engine. 

NOTE: Available at http://www.sun.com/software/sge/ . 

[i.lO] GLite. 

NOTE: Available at http://glite.web.cern.ch/glite/ . 

[i.ll] Gridsystems. Fura. 

NOTE: Available at http://www.gridsvstems.com/ . 

[i.l2] Foster, I.: Globus Toolkit Version 4: "Software for Service-Oriented Systems". In: H. Jin, D.A. 

Reed, W. Jiang (eds.) NPC, Lecture Notes in Computer Science, vol. 3779, pp. 2-13. 
Springer (2005). 

[i.l3] Streit, A., Erwin, D., Lippert, T., Mallmann, D., Menday, R., Rambadt, M., Riedel, M., Romberg, 

M., Schuller, B., Wieder, P.: "UNICORE - From Project Results to Production Grids". 
CoRR (2005). 

[i.l4] Foster, I., Grimshaw, A., Lane, P., Lee, W., Morgan, M., Newhouse, S., Pickles, S., Pulsipher, D., 

Smith, C, Theimer, M.: "OGSA Basic Execution Service Version 1.0", GFD-R.108. Open Grid 
Forum (2008). 

[i. 15] ETSI ES 202 553: "Methods for Testing and Specification (MTS); TPLan: A notation for 

expressing Test Purposes". 

3 Definitions and abbreviations 

3.1 Definitions 

For the purposes of the present document, the following terms and definitions apply: 

deployment manager: entity which converts deployment information into infrastructure specific service calls or 
commands to perform resource reservation and application deployment 



3.2 



Abbreviations 



For the purposes of the present document, the following abbreviations apply: 



AD 

ADL 

CCS 

DD 

GCM 

ICS 

IPT 

LSF 

PBS 

SGE 

SUT 



GCM Application Descriptor 

Architecture Description Language 

Compute Cluster Server 

GCM Deployment Descriptor 

Grid Component Model 

Implementation Conformance Statement 

Internet Protocol Testing 

Load Sharing Facility 

Portable Batch System 

Sun Grid Engine 

System Under Test 
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TP 


Test Purpose 


VN 


Virtual Node 


XML 


Extensible Markup Language 



Testing framework for the Grid Component IVIodel 



4.1 



Introduction 



EG 202 237 [i.l] describes a generic approach to interoperability testing for Internet Protocol Testing (IPT). In this 
guide, interoperability testing is defined as the "activity of proving that end-to-end functionality between (at least) two 
communicating systems is as required by the base standard(s) on which those systems are based". The document 
introduces the interoperability testing process that includes descriptions of basic interoperability concepts, 
interoperability test architectures, and interoperability test models. In addition, the process for the development of 
interoperability test specification and execution are presented. 

In the present document, the development of an interoperability grid testing framework is described which has been 
derived from the generic approach to interoperability testing [i.l]. This grid interoperability testing framework is based 
on the ETSI GCM standards [i.2], [i.3] and [i.4]. The GCM standards were not meant to standardize interfaces but to 
abstract a common set of properties from existing interfaces. In addition, it standardizes the use of these proprietary 
interfaces towards the respective platforms. This approach has been chosen to allow infrastructure providers not to 
change their interfaces for a deployment of applications. 

The main focus of the testing framework is to validate that properties described in the GCM DD and AD are 
implemented by the equipment under test. This equipment includes one or more infrastructures as well as a GCM 
deployment manager as shown in the abstract architecture shown in figure L 

Example test purposes and test descriptions are presented which have been developed based on this framework. Due to 
the proprietary nature of deployment interfaces the testing framework is currently envisioned to primarily serve as the 
basis for development of informal test descriptions suitable for manual execution of tests, e.g. at interoperability events 
such as ETSI Plugtests'"^. However, test descriptions may also be used as a basis for test case development, i.e. an 
automated testing framework. 



4.2 Grid Component IVIodel 



The Grid Component Model (GCM) standard contains three parts: the GCM Application Description [i.2], GCM 
Interoperability Deployment [i.3], and GCM Fractal Architecture Description Language (ADL) [i.4]. A generic GCM 
test architecture which focuses on the GCM Application Descriptor (AD) and Deployment Descriptor (DD) has been 
developed as shown in figure 1 . Here, the user is assumed to provide a (test) application, a GCM DD XML file, as well 
as optionally a GCM AD XML file. 

The GCM DD describes resources requested from one or more different infrastructures for an application. The GCM 
DD is converted by the deployment manager into the invocation of specific infrastructure services or commands. This 
conversion process should be done in an automated manner by a deployment manager but may need to be performed 
manually if the use of the infrastructure interface has not yet been standardized in [i.3]. The GCM DD is mapped to 
resources of the specified infrastructure(s), and then used to deploy and establish a communication layer, called the 
GCM infrastructure, which is used for application deployment and execution. Input and output data servers can be used 
to store input and/or output data of GCM applications independent of the infrastructure on which it runs. Data can be 
accessed remotely or locally. 



£75/ 



ETSI TS 102 786 V1.1.1 (2009-10) 



j GCM AD \ 



references 



^ GCM DD 



converts 



converts 



Deploymenf Manager 



builds 



starts and,-'' 
gets virtual nodes 



Input/Output 
data server{s) 




offers T 
Infrastructure X 




accesses 



Figure 1 : GCM Architecture 

A GCM AD describes the requirements of an application from an underlying infrastructure, e.g. how Virtual Nodes 
(VNs) required by the application are mapped to resources defined in the GCM DD it references. In addition, an 
application may access data servers to read input data and write output data from locations specified in the GCM AD. A 
GCM DD describes resources that are expected to be available and provided by one or more infrastructures. Note that 
the resources requested in a GCM DD are often only a subset of all resources available in one or more infrastructures. 
The resources have to be accessible either in a direct or indirect manner. While infrastructures with indirect resource 
access offer a service that contacts a job scheduler and grants access to resources, infrastructures with direct resource 
access perform the deployment on the resources without any manager. 

Examples of infrastructures with indirect resource access include clusters and/or grid middleware. Examples of 
infrastructures with direct resource access include desktop computers and cloud computing systems. A set of desktop 
computers may also be collected to form a group infrastructure. 

The GCM DD standard contains already a number of standardized mappings to a number of different commercial as 
well as open source infrastructures: 

• Infrastructures with indirect resource access: 

Local resources manager including IBM LoadLeveler [i.5]. Platform Load Sharing Facility (LSF) [i.6], 
Microsoft Compute Cluster Server (CCS) [i.7]. Portable Batch System (PBS) [i.8] and Sun Grid Engine 
(SGE) [i.9]. 

Grid infrastructure including gLite [i.lO], Fura [i.ll]. Globus [i.l2] and Unicore [i.l3]. 

• Infrastructures with direct resource access: 

Desktop computer including MS Windows and Linux. 

Cloud computing system including Amazon EC2. 

Input and output data servers are used to store input and/or output data of GCM applications independent of the 
infrastructure on which it runs. Input and output data servers can be access independently using a supported file access 
protocol such as http, ftp, sftp or file. Depending on the protocol, the data is accessed remotely or locally. 



4.3 



Goals 



The main purpose of the tests is the assessment of the standardized GCM Deployment Descriptor (DD) and Application 
Descriptor (AD). The general test objective is to check that applications can be deployed and executed on a given 
infrastructure based on the information provided in GCM DD and AD. An infrastructure can either provide direct or 
indirect resource access. To access an infrastructure, its protocol need to be followed as specified in [i.3]. 
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For a classification of functionalities that may be provided by a System Under Test (SUT), we define compliance levels 
as follows: 

Compliance by the infrastructure: 

1) An infrastructure does not support properties described in GCM AD and DD. 

2) An infrastructure supports properties described in GCM AD and DD but are converted in a manual manner. 

3) An infrastructure supports properties described in GCM AD and DD and are converted in an automated 
manner. 

Compliance by the deployment manager: 

4) Multiple infrastructures support fulfill level 1 . 

5) Multiple infrastructures support fulfill either level 1 or level 2. 

6) Multiple infrastructures support fulfill level 2. 



4.4 



Test architecture 







^^^ ■ ■ 
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Figure 2: A test architecture for GCIVI-based deployment 

An example test architecture is shown in figure 2. The System Under Test (SUT) consists of the Deployment Manager 
and one or more infrastructures. The different types of entities that compose the means of testing handle provision of 
GCM DD and AD files to the deployment manager associated with the infrastructure to be tested, the evaluation of 
responses from the deployment manager, analysis of the output produced by the application, monitoring of the 
processing ongoing in the infrastructure during the execution of tests, e.g. the number of processors involved in a 
computation, the interface(s) between deployment manager, each infrastructure and the input/output servers. 

NOTE 1 : The interfaces for supplying GCM DD and AD to a deployment manager have not yet been standardized. 

NOTE 2: This testing architecture can also be used to access other standards related to the deployment and 

execution of applications on grid or cloud infrastructures, e.g. a OGSA-BES [i.l4] web service based 
interface between the deployment manager and the infrastructure. 

For each test, a specific test application is used to assess the test purpose which is based on the GCM standard. This 
application should be parameterizable to allow its reuse across multiple tests. 

NOTE 3: It is also assumed that all applications associated with tests already physically reside in the SUT. 
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4.5 Test suite structure and test purposes 

4.5.1 Test suite structure 

The test suite structure should be based on the requirements specified in GCM DD and AD standards. Firstly, test 
groups should distinguish tests to be relevant to GCM DD vs. AD. For the GCM DD groups should differentiate 
between indirect and direct resource access. For the GCM AD groups should differentiate between virtual nodes and 
data location. 

4.5.2 Test purposes 

A test purpose (TP) specifies how a requirement specified in a specification can be assessed in a given test architecture. 
Therefore, each TP should at least include a reference to the clause in a specification where the requirement assessed in 
the test is stated in the standard. In addition, a TP should be dedicated to one aspect of a specific requirement or concept 
defined in a GCM standard. Each test purpose should be unique and have an identifier reflecting its place in the test 
suite structure. The formulation of the test purpose should be based on the writing style recommended in [i.l5]. 

Specification of test purposes for GCM is not trivial since the GCM standard defines the specification of deployment 
information but not an interface for deployment. In case of GCM DD, test purposes should be specified on interface 
parameters which are common in standardized GCM mappings to different infrastructures. A test purpose should 
however not be specific to a single mapping. Other areas for test purpose specification are the assessment of different 
resource access methods (including direct, indirect and bridge) as well as requests for different number of processors 
and/or virtual machines from one or more infrastructures. In table 1 an example TP for GCM DD is depicted. 

Table 1 : Example GCM test purpose "Single processor with direct resource access" 



TPID: 


TP GCM DD DA PA 001 


Clause Ref: 


TS 102 827 [i.3], clause 7.1 


SUT role: 


Deployment Manager and Infrastructure 


Summary: 


Ensure that an infrastructure with direct resource access provides a single processor 
as specified in the GCM DD 



In the case of GCM AD, GCM DD information which is referenced should be assumed to be correct and tested. Test 
purposes here should focus on aspects and parameters only specified as part of the AD, e.g. handling of virtual nodes 
and input/output data location. 



4.5.3 Test descriptions 



A test description is a detailed but informal specification of the pre-conditions and test steps needed in order to cover 
one or more given test purposes. A test description shall contain the following information: 

• Identifier: 

Each TD has a unique identifier that relates a test to its group and sub-group. 

• Summary: 

The TD summary clarifies the purpose of a test. The summary of every TD should be unique. 

• Configuration: 

This field references the test configuration required for the test execution. 

• Specification References: 

One or more clause references to the standard based on which the test purpose has been specified. 

• Test application: 

A reference to the test application which is required to execute this test. In case, the application is 
parameterized the pre-conditions should state constraints on these parameters. Examples for GCM test 
applications include: 

Single process batch job. 

Parallel job. 
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Virtual Node GCM Application. 
Data Manipulation GCM Application. 

• Pre-test conditions: 

A list of all conditions that have to be fulfilled prior to the execution of a test. These conditions should also 
help to identify the standardized features that should be supported by the equipment referenced in the test 
configuration, i.e. to check if a test is applicable for a given equipment. 

Common types of pre-conditions include: 

GCM descriptor content: GCM DD and/or AD content must fulfill certain requirements. 

Infrastructure: type of resource access, features required to be supported (e.g. specification of all clock 
time), and available amount of resource (e.g. at least 2 processors). 

Test application parameterization: number of processes, process execution time, etc. 

• Test sequence: 

Description of stimuli and observations in a numbered order which expected to be performed by an end user 
on interfaces offered by the SUT. Each stimulus should be followed by at least one observation. 

An example TD for the GCM DD is shown in table 2. This TD describes a test to check if an infrastructure with direct 
resource access provides a single processor as specified in the GCM DD. 

NOTE: Test descriptions for GCM DD should be specified independent of GCM AD. 

Table 2: Example GCM test description "Single processor with direct resource access" 



Interoperability Test Description | 


Identifier: 


TD GCM DD DA PA 001 


Summary: 


Ensure that an infrastructure with direct resource access provides a single processor 
as specified in the GCM DD 


Configuration: 


Single Infrastructure or single Infrastructure with a bridge 


Specification 
References: 


GCM DD clause 7.1 


Test Application: 


Single process batch job 




Pre-test 
conditions: 


• Infrastructure provides direct resource access. 

• GCM DD contains a direct group description with hostList containing one host 
and host description with hostCapacity=i for the infrastructure. 

• Infrastructure has a processor available for use. 


^^^r- ^^V 


Test Sequence: 


Step 


Description 


1 


User loads the GCM DD and starts the test application on the infrastructure 
using the deployment manager 


2 


Verify that the infrastructure has created and executed the process 


3 


Verify that returned application output is correct 



4.5.4 Test execution 

Test execution requires the evaluation of the applicability of each test (description) to all of the equipment part of the 
SUT. To speed up this process a GCM Implementation Conformance Statement (ICS) should be established to allow 
infrastructure providers to specify supported features prior to a test execution and support automatic test selection. 
PIXIT can be used capture infrastructure specific aspects of a GCM DD such as the access details to an infrastructure, 
resource identifiers, etc. 

A test should be executed if all of its pre-conditions have been ensured. 

A test should not be executed and recorded as being not applicable if any of its pre-conditions are not met by the one (or 
more) equipment part of the SUT. 
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